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f rW "flfo Invention 

This invention relates generally to a distributed computer system and more specifically to 
event handling procedures in a distributed computer system. 
Fflr kfirnnnd of the Inv ention 

Distributed computer systems are systems in which the programming and data that the 
computer operates on are spread out over more than one computer, usually over a network with 
many computers. One conventional method for organizing a distributed system uses the 
client-server model, in which one part of the distributed system (the client) asks for some service 
from another part of the distributed system (the server). The server responds and then handles 
the client's request appropriately. 

Under the elien.-server model, when the client encounters a procedure located at the 
server, the procedure may he implemented using some form of a Remote Procedure Cal. (RPC): 
the client informs the server that i, would like the procedure executed using certain parameters, 
the server executes the procedure, and the server returns the results of the execution to the client. 

An alternative method for organizing distributed systems uses the event/notification 
mode,. In this model, a "listener" process, which is interested in the occurrence of an even, 
within another entity within the system, registers its interest with a second process, a "notifier" 
process, designed to monitor such events. When the even, occurs, the no.if.er notifies the 
list ener or notifies ano,her process (i.e., a third party process) designated by the listener when 
registering with the nofifier. After receiving the even, no,ifica,ion, ,ha, notified entity (i.e., the 
,; s ,ener or ,he third party listener, as the case may be) may execute a function designated in the 
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notification and present in the address space of the notified process. In the event-no.ifieation 
ra0 de., there is no sin 6 l= point of control as in the ciient-server mode!; instead, control is 
determined by notifications sent in response to the occurrence obsignated events. 

,„ a non-distributed system, the association of an even, with the function to be run in 
response to the notification, caUed a callback function, is straightforward, sine the even,, the 
Ustcner, and the caUback function are ah in a single address space. Life is no. so simp.e in a 
distributed system. The listener may be in one address space or physical machine, the even, may 
be general inasecond address space or physical machine (e.g., me address space of Ore nofifier 
or an address space thatthe notifier monitors), and the notification may be senttoathird address 
space or physical machine (e.g., the address space of ,he mird party process). This makes it 

in an appropria.emanner.as.he.won.aybeseparated and, possibly, ntigh. no, even know of fine 
„.hefse X is,ence. For example, Ute lis,ener,,o ensure Uta.dte correc. func.ion is e X ecu,ed by fine 
third party.mus, keep .rack ofwhe.her the function is availab.e in the third party's address space. 
TMscanbepanicularlyburdensome^^ 

Unird parry's address space, as alt ,he po.en.ial distributed users of ,he new function mus, be 
updated on the status of the new function. 

Thus, there is a need in the an to more effectively handle distributed programs 
collaborating using the event-notification mode,. Thisneed is particularly poignant in systems in 
W hich fine re q ues,in 8 en,i,yis no, abletoeasily verify the sta.us of a function present at the 
executing entity. 



= ls^eCAS. Hf.SOEP.SON, 
rAKA30tt\ CARRsTT. 

6 Dcssek.i.i.P- 

,300 : tracer, n.w. 



I ?rm m arv nf lh * '"vention 



Object and advantages of .he invention will be sc. forth in part in .he description which 
follows, and in part will be obvious from the description, or may be learned by practice of the 
invention. The objects and advantages of .he inven.ion will be realized and attained by means of 
the elements and combinations particularly pointed out in me appended claims. 

To achieve the objects and in accordance with the purpose of the invention, as embodied 
and broadly described herein, a firs, aspect consistent with the present inven.ion includes a 
m ethod for controlling program execution in a dis.ribu.ed compu.er sys.em comprising .he s.eps 
of: (1) registering in.eres. in an occurrence of an even, in the distributed compu.er sys.em, the 
regis.ra.ion of in.eres. including information identifying the occurrence of ft. event, an identifier 
of a software entity in the distributed system, and a first object including a process and parameter 
da.a corresponding to the process; (2) monitoring a. least a portion of the distributed computer 
system for the occurrence of the registered even, (3) notifying the software entity identified in 
fte regis.ra.ion of in.eres. when .he even, occurs, the notification including a copy of the first 
object and an identification of fte event fta. occurred; and (4) execu.ing methods contained 
within the first object in response to fte notifying step. 

To achieve fte objects and in accordance with fte purpose of the invention, as embodied 
and broadiy described herein, a second aspect consistent with the present invention includes a 
protocol for controlling the execution of processes in a dis.ribu.ed compu.er sys.em, the pro.oco! 
comprises a number of s.eps, including: (1) receiving a regis.ra.ion of in.eres. in an even, that is 
expected to occur in the distributed compu.er system, the registration includ.ng an identifier of a 
software en.uy in .he distributed system and a first object, .he firs, object including computer 
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instructions for Performing a process and parameter da,a corresponding to the process; (2) 
rnonuoring the distributed system for the occurrence of the registered event; and (3) noticing ,he 
software entity identified in the registration of interest when the event occur, the notification 
including a copy of the first object and an identification ofthe event that occurred. 

A third aspect of the present invention is a computer system comprising memories havmg 
Hrst. second, and third virtua. machines, respectively. The second virtual machine executesa 
pr „cess that receives, from the firs, virtual machine, a registration of interest in an event and 
^amessagcintesponsetoUteoocu.eneeoftheevent.Ute Ration ofinteres. and the 
message including anobject. The third vinua, machine receives the message and executes 

methods contained within the object. 

Additional aspectsofdtepresentinven.ion.re.atcd to the firs, aspect, arc directed toa 
computer readab.e medium and a computer data signal embodied in a carrier W ave. 

Pr j nf p.crp ptinn of the D rawings 



The accompanying drawings, which are incorporated in and constitute a pat of dtis 
.pecification.Ulus.tcseveralembodimen. — widt mis invention and, togemer with fire 
description, he.pexplain the princip.es of the invention. In the drawings, 
Fig. 1 is a diagram of an exemplary distributed system; 

Fig .2isadiagramofan exemplary computer within the exemplary distributed system; 
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Fig. 3A is a flow chart illustrating 



methods consistent with the present invention for 



notifying and transporting 



behavior in an event based distributed system; 
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Fig. 3B is a table showing exemplary event noti: 



fication requests stored in a computer 



memory; and 



distributed computing system containing three 



Fig. 4 is a block diagram illustrating a 
e S c m p 1 a ty co m pu.erp 1 3,for m sco„„ec,cd,oo„ea„o l h=rv iaa „e t wo r , 
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I characteristics of this environment, including the security offered by it and the strong typing 
j provided by it. The Java programming environment is more clearly described in Jaworski, to 
1, 1 pro per's Guide . Sams.net (1997), which is incorporated herein by reference. 



In the exemplary distributed system, different computers and devices are federated into 
what appears to the user to be a single system. By appearing as a single system, the exemplary 
distributed system provides the simplicity of access and the power of sharing that can be 
provided by a single system without giving up the flexibility and personalized response of a 
personal computer or workstation. The exemplary distributed system may contain thousands of 
devices operated by users who are geographically disperse, but who agree on basic notions of 
trust, administration, and policy. 

Within the exemplary distributed system are various logical groupings of services 
provided by one or more devices, and each such logical grouping is known as a Djinn. A 
"service" refers to a resource, data, or functionality that can be accessed by a user, program, 
device, or another sen-ice and that can be computational, storage related, communication related, 
or related to providing access to another user. Examples of services provided as part of a Djinn 
include devices, such as printers, displays, and disks; software, such as applications or utilities; 
information, such as databases and files; and users of the system. 

Both users and devices may join a Djinn. When joining a Djinn, the user or device adds 
zero or more sen-ices to the Djinn and may access, subject to security constraints, any one of the 
services it contains. Thus, devices and users federate into a Djinn to share access to its services. 
The sen-ices of the Djinn appear programmatically as objects of the Java programming 
environment, which may include other objects, software components written in different 
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programming .anguages, or hardware devices. A service has an interface defining the operations 
thatcan be requested of that service, and the type of the service determines the interfaces that 

make up that service. 

Fig. 1 depicts the exemplary distributed system 100 containing a computer 102, a 
computer .04, and a device 106 interconnected by a network .08. The device 106 may be any of 
anumberof devices, such as a printer, fax machine, storage device, computer, or other devices. 
The network ,08 may be a .oca, area network, wide area network, or the Interne.. A.though on.y 
lwo computers and one device are depicted as comprising the exemp,ary distributed system ,00, 
one skified in the art wi„ appreciate that the exemp.ary distributed system 100 may include 
additional computers or devices. 

Fig. 2 depicts the computer ,02 in greater detail to show a number of the software 
componentsofmeexemplarydistributedsystem.OO. One skilled in the art will appreciate ma. 
computer .04 or device .06 may be simuar.y configured. Compu.er .02 inc.udes a memory 
202, a secondary storage device 204, a central processing unit (CPU) 206, an input device 208, 
a„davideodisp.ay2,0.Thememory202inc,udesa.ookupservice2,2,adiscoveryserver2,4 

.nd a Java™ runtime sys.em2,6. The Java runtime system2,6 indudes the Java™ remote 
method invocation system (RMI, 2.8 and a Java™ virtua, machine 220. The secondary storage 
device 204 includes a JavaSpace™ 222. 

AS mentioned above, the exemplary distributed system 100 is based on the Java 
programming environment and thus makes use of the Java runtime system 2,6. The Java 
runtime system 2,6 inc.udes the Java™ API, aUowing programs running on top of the Java 
runtime system to access, in a p.atforrn-independent manner, various system functions, inciudmg 
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;j windowing capabilities and networking capabilities of the host operating system. Since the Java 
1 API provides a single common API across all operating systems to which the Java runtime 
|| system 216 is ported, the programs running on top of a Java runtime system run in a platform- 
independent manner, regardless of the operating system or hardware configuration of the host 
platform. The Java runtime system 216 is provided as part of the Java™ software development 
kit available from Sun Microsystems of Mountain View, CA. 

The Java virtual machine 220 also facilitates platform independence. The Java virtual 
machine 220 acts like an abstract computing machine, receiving instructions from programs in 
the form of byte codes and interpreting these byte codes by dynamically converting them into a 
form for execution, such as object code, and executing them. RMI 2 1 8 facilitates remote method 
invocation by allowing objects executing on one computer or device to invoke methods of an 
object on another computer or device. Both RMI and the Java virtual machine are also provided 
as part of the Java software development kit. 
>u The lookup service 212 defines the services that are available for a particular Djinn. That 

^^lllmzy be more than one Djinn ^consequently, more than one lookup service within the 
exemplary distributed system 100. The loo\p service 212 contains one object for each service 
within the Djinn, and each object contains varies methods that facilitate access to the 
corresponding service. The lookup service 212 anks access are described in greater detail in 

co-pending U.S. Patent Application No. , en\tled "Method and System for 

Facilitating Access to a Lookup Service," which has previously been incorporated by reference. 
The discovery server 214 detects\hen a new device is added to the exemplary 
^Z^^^siZ^ system 100. during a process kn\as boot and join .rediscovery, and when such a 
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new device is detected, thediscovery server passes a reference to the lookup semce 212 to the 
new device, so that the new device may register its services with the lookup service and become 
a member of the Djinn. After regisVation, the new device becomes a member of the Djinn, and 
as a result, it may access all the service^ contained in the lookup service 212. The process of 
boot and join is described in greater detaiKin co-pending U.S. Patent Application No. 

, entitled "Apparatus and Method lor providing Downloadable Code for Use in 

Communicating with a Device in a Distributed ^stem," which has previously been incorporated 
by reference. 

The JavaSpace 222 is an object repository used by programs within the exemplary 
distributed system 100 to store objects. Programs use the JavaSpace 222 to store objects 
persistently as well as to make them accessible to other devices within the exemplary distributed 
system. JavaSpaces are described in greater detail in co-pending U.S. Patent Application No. 
08/971,529, entitled "Database System Employing Polymorphic Entry and Entry Matching," 
assigned to a common assignee, filed on November 17, 1997, which is incorporated herein by 
reference. One skilled in the art will appreciate that the exemplary distributed system 100 may 
contain many lookup services, discovery servers, and JavaSpaces. 

Although systems and methods consistent with the present invention are described as 
operating in the exemplar)' distributed system and the Java programming environment, one 
skilled in the art will appreciate that the present invention can be practiced in other systems and 
other programming environments. Additionally, although aspects of the present invention are 
described as being stored in memory, one skilled in the art will appreciate that these aspects can 
also be stored on or rend from other types of computer-readable media, such as secondary storage 
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devices, like hard disks, floppy disks, or CD-ROM; a carrier wave from the Internet; or other 
forms of RAM or ROM. Sun, Sun Microsystems, the SunLogo, Java, and Java-based trademarks 
are trademarks or registered trademarks of Sun Microsystems Inc. in the United States and other 
countries. 

Fig. 3 A is a now chart illustrating methods consistent with the present invention for 
notifying and transporting behavior in an event-based distributed system. Listener objects 
interested in the occurrence of a certain even, must register an interest in the event. In particular, 
for each interest to be registered, the registering object sends a message to a notifier process, 
which is preferably an object containing procedures for monitoring the occurrence of the even, 
(step 301). Generally, the notifier will be present in the virtual machine in which the event is 
expected to occur. The registration message includes: (1) implicit or explicit information 
identifying the event mat is to be monitored, (2) information identifying the object that is to be 
notifted when the even, occurs, and (3) an object, or a reference to an object, Ota. is to be passed 
to the notified object when the even, occurs. The notifier may use a table to store information 
relating to registration messages it has received, such as the table shown in Fig. 3B. The notifier 
monitors its system or the network for the occurrence of the event (step 302). When the event 
occurs (step 303), the notifier notifies the object identifted by the registering object that the 
registered event has occurred (step 304). Consistent with the present invention, the notification 
includes an identification of the event that occurred and the object or reference to the object that 
was passed to the notified object when the event occurred (step 304). 
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An "event" as used throughout this specification can be broadly defined as a change of 
! i system state. An event may be, for example, a timer event, a mouse click event, or a disk access 
jj event. 

jj Fig. 4 is a block diagram illustrating a distributed computing system containing three 

exemplary computer systems 412, 41 3, and 414, connected to one another via a network. 
Computer systems 412-414 are physically similar to computer systems 102 and 1 12. In 
particular, each of computer systems 412, 413, and 4 14 includes a computer platform or machine 
409, 410, and 41 1, respectively, executing a virtual machine 406,-407, or 408. Processes 402, 
403, and 404 reside on their respective virtual machines 406, 407, and 408. Consistent with the 
present invention, computer systems 412, 413, and 414 form a distributed computing system 
using the event/notification model. 

A hypothetical situation illustrating an application of the present invention will now be 
described with reference to the distributed system shown in Fig. 4. Assume that "administrator- 
process 402, executing in virtual machine 406, has the responsibility of monitoring the network 
and reporting potential problems to a human operator-such as a disk drive running out of free 
space. Notifier process 403, executing on virtual machine 407, monitors the system for the 
occurrence of disk drive full events. To register interest in a "DiskFull" event, process 402 
transmits a registration message, called, for example, "Registerlnterest," to notify process 403, 
such as the message: 

Registerlnterest (DiskFull, RefercnceToVM408, SendPage("<parameters> ")). 
In this message, "DiskFull" identifies the event that is to be monitored, "ReferenceToVM408" is 
a reference to the entity that is to be notified in response to the occurrence of the event (in this 
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case, virtual machine 408), and "SendPage" is an object that includes methods and data that, 
when executed on a computer having a telephone connection and the ability to dial a pager 
number, will instruct the computer to page a person at a specified number. 

When the DiskFull event occurs, notifier process 403 notifies the designated entity, 
virtual machine 408, using a generic notify method including the SendPage object, or a reference 
to the "SendPage" object, and an indication that the DiskFull event has occurred. In response, 
virtual machine 408 may create a process such as process 404 that executes the methods 
designated in the SendPage object. The SendPage object will then cause computer system 41 1 to 
page the appropriate technician. 

As mentioned above, the notify method is preferably a generic method, meaning each 
virtual machine and/or computer platform may include its own unique implementation of the 
notify method, although each implementation must meet the specifications required by the 
definition of the generic notify method. 

Because the SendPage object includes, within itself, all required methods and data for 
execution, SendPage is considered to be an object having "closure." By passing an object having 
closure to a designated entity, the object initially registering the object does not have to be aware 
of the functions available at the final destination of the passed object. This is particularly 
advantageous as it allows the object initially registering the event to easily modify the 
functionality of the passed object. For example, if the policy (functionality) of the SendPage 
object is created by an operator at computer 412, it is relatively easy for the operator to modify 
the policy of SendPage by, for example, instructing SendPage to page a backup technician if the 
page is not returned by the primary technician within twenty minutes. 
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Although SendPage was described as having closure by virtue of having all its required 
methods and data encapsulated within the object, other objects may be passed that have closure 
even though code still must be supplied by the executing system. In particular, if the code that 
must be supplied by the executing system implements well known methods that can be 
implemented in any one of a number of acceptable ways, the object may have closure merely by 
referencing the generic method definitions and allow the executing system to provide the precise 
implementation. 

While there has been illustrated and described what are at present considered to be 
preferred embodiments and methods of the present invention, it will be understood by those 
skilled in the art that various changes and modifications may be made, and equivalents may be 
substituted for elements thereof without departing from the true scope of the invention. For 
example, while the foregoing systems and methods have been described with reference to a Java 
based run-time environment, other run-time environments could conceivably be used to 
implement the present invention. 

In addition, many modifications may be made to adapt a particular element, technique or 
implementation to the teachings of the present invention without departing from the central scope 
of the invention. Therefore, it is intended that this invention not be limited to the particular 
embodiments and methods disclosed herein, but that the invention include all embodiments 
falling within the scope of the appended claims. 
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